Planning routes and allocating identifiers to routes in a managed frame-forwarding network

ABSTRACT

A method is provided of planning routes and allocating route identifiers in a managed frame-forwarding network. The network comprises a plurality of nodes interconnected by links, with each node being arranged to forward data frames according to a combination of an identifier and a network address carried by a received data frame and forwarding instructions stored at the node. A first step of the method identifies a sub-set of nodes which are core nodes of the network. The remaining nodes are termed outlying nodes. A spanning tree is then built off each of the identified core nodes, with the spanning tree stopping one link short of any other core node. Each spanning tree defines a loop-free path between a core node at the root of the spanning tree and a set of outlying nodes. Connections are planned between roots of the spanning trees and a different identifier is allocated to each planned connection between a pair of spanning trees.

CROSS-REFERENCE TO RELATED APPLICATION

This is a continuation of U.S. Ser. No. 11/343,996, filed Jan. 31, 2006now U.S. Pat. No. 7,756,035, which is hereby incorporated by reference.

BACKGROUND TO THE INVENTION

For many years now, telecommunications carriers have been deployingpacket-switched networks in place of, or overlaid upon, circuit-switchednetworks for reasons of efficiency and economy. Packet-switched networkssuch as Internet Protocol (IP) or Ethernet networks are intrinsicallyconnectionless in nature and, as a result, suffer from Quality ofService (QoS) problems. Customers value services which are guaranteed interms of bandwidth and QoS.

It is desired to use Ethernet switches in carriers' networks. Use ofEthernet switches in carriers' networks would have the advantages ofinteroperability (mappings between Ethernet and other frame/packet/celldata structures such as IP, Frame Relay and ATM are well known) andeconomy (Ethernet switches are relatively inexpensive compared to IProuters, for example). It would also provide a distinct advantage ofbeing the principal technology used by enterprises that require a widearea network service from a carrier and therefore able to work in anative mode. In view of the above, there have proposals to implement‘Connection-oriented Ethernet’ (CoE) networks where managed trafficpaths (connections) are set up across a network of Ethernet switches incontrast to conventional Ethernet paths where switches decide forthemselves how to forward packets. One such proposal is described in WO2005/099183. In WO 2005/099183 a network manager instructs each Ethernetswitch along a route to store forwarding information. The switch usesthe forwarding information to forward received data frames. A particularcombination of identifiers in a data frame, such as a Virtual Local AreaNetwork Identifier (VLAN ID or VID) and a destination address (DA) areused to differentiate traffic routes.

In a conventional destination-based forwarding scheme such as Ethernet,a single address entry in the bridging table at an intermediate switchis sufficient for all sources to reach a destination if all sources usethe same route. In a network of the type described in WO 2005/099183VIDs are used as route discriminators, allowing multiple routes to befollowed to a destination. This requires a network management system toset up routes across the network, storing ‘state’ in each Ethernetswitch. The state is an entry in the forwarding table at each switchalong a chosen route. Assigning VIDs to different routes in a piecemealfashion can quickly use a large number of different VIDs, requiringswitches in a network to store a large amount of state. VIDs are limitedin number, and excessive VID consumption is deprecated because it limitsthe number available of VIDs available for conventional bridgedoperation on the same infrastructure, if desired. Storing unnecessaryfast-path state also incurs a direct cost penalty on the switches.

The present invention seeks to allocate identifiers in a more optimalmanner.

SUMMARY OF THE INVENTION

A method is presented for planning routes and allocating routeidentifiers to those planned routes in a managed frame-forwardingnetwork. The network comprises a set of nodes interconnected by links,with each node being arranged to forward data frames according to acombination of a route identifier and a network address carried by areceived data frame and forwarding instructions stored at the node. Themethod comprises identifying a sub-set of the nodes which are core nodesof the network. The remaining nodes are called outlying nodes. Themethod then builds a spanning tree off each of the identified corenodes, with the spanning tree stopping one link short of any other corenode. The spanning tree defines a loop-free path between the core nodeat the root of the spanning tree and a set of outlying nodes. The methodthen plans connections between roots of the spanning trees and allocatesa different route identifier to each planned connection between a pairof spanning trees.

This method simplifies the task of establishing diverse connectivityacross a network. In essence, zones of connectivity are constructed (inthe form of local spanning trees) around a set of core nodes acting aswaypoints (selected on the basis of local mesh density) with the domainof each zone hounded by collisions with other waypoints. The criteria bywhich waypoints self elect is chosen such that the number of waypointsis a subset of the nodes in the network and offer a usefulabstraction/summarization of connectivity. One can then consider thechallenge of routing of diverse paths to simply being selecting pathsthat transit non-overlapping set of waypoints.

The method allocates identifiers to routes in a structured manner, whichhas the advantage of minimising the overall number of differentidentifiers that are used. This has the advantages of reducing theamount of state which needs to be stored at individual switches withinthe network.

A richly connected set of core nodes are selected as the waypointsaccording to some criterion, and used as the roots of a set of spanningtrees. Each spanning tree is rooted on one core node, and includes noother core nodes. The use of a spanning tree algorithm defines areliable loop-free path connecting a set of outlying nodes. By stoppingthe spanning tree short of another core node, this should create aspanning tree which serves a ‘sector’ of the network. The spanning treealgorithm does not need to be the Spanning Tree algorithm as defined byIEEE, and can be any other suitable algorithm which has the effect ofsimply connecting nodes, that means connecting all nodes with only oneroute existing between any pair of nodes. It is expected that outlyingnodes will be served by two (or possibly more) spanning trees, whichmeans that the outlying nodes will have routes to two core nodes. It ispreferable that those routes are fully diverse, or as diverse as ispossible. The results of the spanning tree analysis can be reviewed andthe spanning tree can be modified. One way in which the results of thespanning tree can be modified is by modifying the value of a metric(such as link cost) associated with a link which is undesirably routed.

In some networks the identified group of core nodes can include a corenode which connects only to other core nodes, and does not connect toany outlying nodes, i.e. a core node which only serves as a transit nodebetween other core nodes. In this case, no spanning tree is built offthat core node as it would not serve any useful purpose. The step of“building a spanning tree off each of the identified core nodes,stopping one link short of any other core node” will prevent a spanningtree being built off a core node which is only connected to other corenodes.

The use of STs in this way has been found to provide a good basis onwhich to then allocate route identifiers. Routes between nodes arealways bi-directional and co-located. i.e. the forward and reversetraffic is routed along the same path. In conventional Ethernet, becausea VLAN defines a broadcast topology a single bi-directional VID isassociated with a VLAN and is used for both directions of apoint-to-point (p2p) connection, Connection-Oriented Ethernet, being bydefinition point-to-point (p2p), does not need this restriction, and wecan choose to use a different VID for each direction (i.e. a destinationlogically chooses the VID without reference to anyone else). This wecall a ‘uni-directional VID’. A bi-directional VID is used for bothdirections across the network whereas a uni-directional VID is used foronly one direction, and a different VID can be used for the otherdirection. The requirements of allocating VIDs are different dependingon what type of VID is being used. The specific details of the processof allocating identifiers differs for the bi-directional anduni-directional cases. It has been found that where bi-directional routeidentifiers are used, the maximum number of required identifiers scalesas a function of C², more precisely (C·(C−1))/2, where C=number of corenodes. Where uni-directional identifiers are used, it has been foundthat the maximum number of required identifiers scales as a function of(C×D), where D is the number of diverse paths between the roots of STpairs. It has been found that the actual number of required identifierscan be considerably less than these maximum bounds.

A preferred scheme uses a combination of a VLAN ID (VID) and EthernetDestination Address (DA) to identify a route through the network,although other combinations of elements could be used.

Further aspects of the invention provide a network entity which isarranged to perform the above method and a machine-readable mediumcarrying instructions for causing a processor to perform the method ofplanning routes and allocating route identifiers.

Once identifiers have been determined using this method, they aredistributed to switches at network nodes and used as forwardinginstructions at those nodes. Accordingly, a further aspect of theinvention provides a method of generating forwarding instructions for anode in a managed frame-forwarding network.

Further aspects of the invention are defined in the appended claims.Further advantages of the invention will be apparent from the followingdescription.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be described, by way of example only,with reference to the accompanying drawings in which:

FIG. 1 shows an arrangement of Ethernet switches forming a network;

FIG. 2 shows the overall method of allocating route identifiers withinthe network of FIG. 1;

FIG. 3 shows the results of identifying core nodes of the network andconstructing spanning trees off the core nodes;

FIG. 4 shows parts of the network where spanning trees requiremodification;

FIGS. 5A-5C show modification of the spanning trees for a simple networkexample;

FIG. 6 shows the results of modifying the spanning trees shown in FIG.4;

FIG. 7 shows allocation of bi-directional VIDs to the network of FIG. 6;

FIG. 8 shows allocation of uni-directional VIDs to the network of FIG.6;

FIG. 9 shows the use of short cut routes in the network of FIG. 8;

FIG. 10 shows the network and control plane; and,

FIG. 11 shows an example of equipment at each managed switch of thenetwork.

DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 shows an example Ethernet network 5 having a set of nodes(Ethernet switches) interconnected by communication links. As anexample, outlying nodes 21, 22 are interconnected by a link 23.

FIG. 2 shows the steps of a method to plan routes within the network andto optimally assign route identifiers to the routes planned within thenetwork. As a first step 61 of analysing the network, a set of corenodes are identified. In the example network of FIG. 1, a core node isdeemed to be a node which is directly connected to four or more othernodes. It can be seen that the nodes 10, 12 and 13 each meet thiscriterion. In this case of node 13, it is connected to nodes 10, 12, 14and 15. It will generally be found that the core nodes will be majorPoints of Presence (PoP) or Transit points. The criterion for a nodebeing a core node is not restricted to the test described here.Generally it can be a node which is connected to N other nodes (where Nis set at a value which is found to offer good results) and it may bedesirable to select nodes according to an additional criterion oraccording to at least one alternative criterion. As an alternative toautomatically selecting the core nodes by an algorithm, the core nodescan be selected manually or according to design intent, i.e. selectingthose nodes which the planners of the network, when laying out thenetwork, intended to be used for transit. In this example, it isdesirable that node 11 is designated a core node. Although node 11 doesnot meet the strict criterion of “connected to four or more other nodes”the design intent of this node is that it should be used for transit.Therefore, the total set of core nodes in this network is the set ofnodes 10-13. In a network environment where there is a carrier networkand customer (edge) networks, the core nodes are restricted to the nodesof the carrier network.

FIG. 3 shows the next step (62, FIG. 2) of the method. A spanning treeis built off each of the identified core nodes 10-13. One well-known wayof creating a spanning tree is the result of performing the SpanningTree network protocol (STP) as defined in IEEE 802.1d, but any otheralgorithm can be used that forms a simply-connected tree (defined ashaving only one path between any pair of nodes). This is a known featureof Ethernet technology where each node examines how it is connected toother nodes in the network and establishes the best, loop-free, route toneighbouring nodes. The results can be expressed as a tree diagram withthe starting node at the top and branches leading off representing thehops needed to reach other nodes. The spanning tree algorithm isinfluenced by features of a link such as link cost, where the link costis a notional weighting value which influences the computation. In thisinvention, each spanning tree is stopped one hop before reaching anothercore node. As an example, it can be seen that the spanning tree builtoff core node 10 (green) stops one hop short of core node 11 (purple)and similarly the spanning tree built off core node 11 (purple) stopsone hop short of core node 10 (green). For useful results, core nodesshould generally be chosen so that the spanning trees built off themonly cover a ‘sector’ of the network, which is generally directedoutwards from the core nodes.

The first pass of the method described above results in the set ofspanning trees shown in FIG. 3. The results of this first pass aregenerally successful but there are a few places with non-ideal routing.This is to be expected when applying a general technique to a realnetwork topology. The main concern is that where a node is served by twocore nodes (known as ‘dual-homing’) the two routes to the node should bediverse, i.e. paths arrive at the node from different directions.Looking at FIG. 4, in the places ringed 31-34 in FIG. 4 the results areuseful. Taking the node 32, it is dual-horned to core nodes 10 and 11via paths that arrive at the node 32 from different directions. However,in a small number of cases the results of the automatic spanning treesare undesirable. These are shown as 35, 36. Node 35 is dual-homed tocore nodes 10 and 11 but the routes from the core nodes 10, 11 to thenode 35 are routed in the same direction, i.e. the link segments areco-directed. Similarly, node 36 is dual-horned to core nodes 12 and 13but the routes from the core nodes 12, 13 to the node 36 areco-directed. To improve the effectiveness of the spanning trees, allspanning tree segments travelling in same direction along a link areidentified (i.e. the situations shown as 35, 36) and the cost of theco-directed links is increased. The Spanning Tree algorithm is performedagain for each of the core nodes which serve the node having theco-directed links. The increased cost of the problematic link shouldforce one of the spanning tree algorithms to find an alternative route.This method is a basis of finding diversely routed paths betweenperiphery nodes and the core network.

FIGS. 5A-5C show a sequence of steps for modifying a Spanning Treeoperation. For clarity, a simple network topology representing part ofan overall network is shown in FIG. 5A. FIG. 5A shows two core nodes 50,51 (other links to the core nodes are not shown) and a set of four nodes52-55 which are interconnected by links 56 as shown. FIG. 5B shows theresults of performing a Spanning Tree operation on each of the corenodes 50, 51. Core node 50 has a simple ST with the single branch 57which connects (in order) to nodes 52, 53, 55 and 54. Core node 51 has aST with two branches. A first branch 58 connects to nodes 52 and 53while a second branch 59 connects to nodes 54 and 55. It can be seenthat node 53 is dual-homed to core nodes 50, 51 but that it is served bylinks of Spanning Trees 57, 58 which are co-directed. Thus, if a failureoccurred in the transport network on link 60, node 53 would fail to beconnected to both of core nodes 50, 51. FIG. 5C shows how this problemcan be corrected. The cost of link 60 is increased. This forces theSpanning Tree algorithm to adapt so that branch 58 no longer extends tonode 53. Instead, branch 59 now extends to node 53 via node 55.

FIG. 6 shows the results of modifying the spanning trees of the networkshown in FIG. 4. At node 35, the previous path into node 35 from corenode 10, via link 40, is rerouted via node 32 and link 41 as a result ofincreasing the cost of link 40. Similarly, at node 36, the previous pathinto node 36 from core node 12 via link 42 is rerouted via node 22 andlink 43 as a result of increasing the cost of link 42. The optimised setof spanning trees are used in the next step (63, FIG. 2) of the method.

As an alternative to automatically optimising the spanning trees, theycan be manually modified after their initial computation, or iterativelyoptimised by a repeated sequence of automatic calculation and manualmodification.

FIG. 7 now considers planning the connections between core nodes. Anend-to-end (e2e) route between two outlying nodes of the network can beseen, in general, as a concatenation of:

-   -   (i) a segment between the first outlying node and a core node;    -   (ii) a segment between core nodes of the core network; and    -   (iii) a segment between a core node and the second outlying        node.

From the analysis above, it has been shown that one VID can uniquelyidentify a route between all nodes on a spanning tree (ST) and arespective core (root) node of that spanning tree. The core (root) nodeacts as a waypoint for traffic on that spanning tree. The requirementsof the core network will now be considered based on the topology shownin FIG. 7. Each route between a unique pair of core nodes requires aunique VID. Some of these routes are a single hop, as in the case ofcore nodes 11-10, 11-12, 10-13, 10-12 and 12-13. Due to the core networktopology, the route between core nodes 11-13 requires two hops andpasses via core node 12. (There is also a second possible two-hop routebetween core nodes 11-13 which passes via core node 10 which will beconsidered later.)

All connections are always bi-directional and co-routed, i.e. theforward and reverse traffic is routed along the same path. Inconventional Ethernet, because a VLAN defines a broadcast topology asingle bi-directional VID is associated with a VLAN and is used for bothdirections of a p2p connection. Connection-Oriented Ethernet, being bydefinition point-to-point (p2p), does not need this restriction, and wecan choose to use a different VID for each direction (i.e. a destinationlogically chooses the VID without reference to anyone else). This wecall a ‘uni-directional VID’. A bi-directional VID is used for bothdirections across the network whereas a uni-directional VID is used foronly one direction, and a different VID can be used for the otherdirection. The requirements of allocating VIDs are different dependingon what type of VID is being used.

Firstly, the requirements of bi-directional VIDs are considered. A pairof spanning trees are stitched together across the core. Core nodes willtypically have full (or near-full) mesh connectivity between them, butthis is not a necessary requirement. Using a bi-directional VLAN tag,each ST rooted on a core node must be coupled with every other ST foruniversal e2e connectivity. For the four core nodes shown, this requires3+2+1=6 VIDs as shown in the table in FIG. 7. One unique VID isallocated to each ingress/egress ST pair. In general, for a core havingC core nodes:

${{Number}{\mspace{11mu}\;}{of}\mspace{14mu}{VIDs}} = \frac{C\left( {C - 1} \right)}{2}$

In this manner, an end-to-end route between outlying nodes via any corenode(s) to which they are joined by spanning trees can be uniquelyidentified by a relatively small number of VIDs. In the above example,only 6 VIDs are required for basic connectivity. The term ‘basicconnectivity’ means providing a single connection between every pair ofcore nodes. In the example shown in FIG. 6, there are two possibleroutes between core nodes 11, 13—one route passing via node 12 and theother route passing via node 10. It is possible to distinguish betweenthese routes by using an additional VID. This is a further VID over andabove the number just defined for basic connectivity.

With reference to FIG. 7, suppose we wish to establish two distinctroutes across the core from the peripheral nodes on purple and green STsto those on red and blue STs:

-   -   the route 11        12 has no core nodes in common with the route 10        13, and so communication is resilient to any single point of        failure    -   but the routes 10        12 and 11        13 via 12, although having disjoint terminations, all pass        through core node 12 which is therefore a single point of        failure.

Earlier, we described methods to ensure that overlapping STs rooted ondifferent core nodes were not co-directional, and so achieved disjointroutes between each outlying node and more than one core node if thiswas possible; this combined with this selection of disjoint routesacross the core makes this resilience to any single point of failure anend-to-end property for all nodes.

So far, each core node has been described as having one ST. It ispossible to build multiple STs per core node for traffic engineeringpurposes. If these STs intersect away from the core node, this givesrise to a loop. Therefore it is necessary to provide a different VII)for each intersecting ST. Disjoint (i.e. non-overlapping) STs from asingle core node can use a single VID as the destination node addressalone will be unambiguous at any point in either ST.

Next, the requirements of un-directional VIDs are considered. Forclarity, the network of FIG. 7 has been simplified to that shown in FIG.8 by replacing the group of nodes 16 with a single node 17. This newnode 17 has the same connectivity to the core nodes as the group ofnodes 16. The method is illustrated with respect to the topology of FIG.8. This topology has just four core nodes 10-13. To start with, a VID isassigned to each of the STs in turn, which has the meaning that trafficto be routed off the core does so via the core node at the root of thatST. So:

-   -   traffic, for the <green> ST leaves the core via node 10;    -   traffic for the <purple> ST leaves the core via node 11;    -   traffic for the <red> ST leaves the core via node 12; and,    -   traffic for the <blue> ST leaves the core via node 13.

It is straightforward to observe that if the four VIDs have differentnumerical values, then no ambiguities in route designation can occur.Whatever the ingress side topology, a unique single route to an endpointcan be defined passing through the core node which roots the selectedST.

Next, we can state that if a pair of core nodes have STs with any nodesin common, they must have different VIDs because otherwise there wouldclearly be ambiguity, viewed from the ingress side, by which ST/codenode the core should be egressed. By the same reasoning, ST/core nodeswith no common nodes on their STs can adopt the same VID, because therespective STs have no MAC addresses in common and thus the MAC addresscan distinguish the routing to the destination.

In the example of FIG. 8, the VID associated with <green> egress canhave numerically the same value as either <blue> or <red> (but not<purple>). The VID associated with <purple> can also have numericallythe same value as either <blue> or <red>. The VIDs associated with <red>and <blue> must be mutually different, as must the VIDs associated with<green> and <purple>, because each pair of STs has nodes in common.

End-to-end routes can now be formed. Each VID defining an egress nodeand its associated ST defines a single route across the core betweennodes on an ingress ST and its associated core node. This is because ofthe requirement that both directions must be co-routed, thus thebi-directional route must be formed by joining a pair of spanning tressacross the core. So, the connection of <purple> to <red> (andvice-versa), and <green> to <blue> (and vice-versa) defines one pair ofbi-directional routes across the core. Similarly, the connection of<purple> to <blue> (and vice-versa) via, for example, node 12, and<green> to <red> (and vice-versa) defines another pair of bi-directionalroutes across the core. Both are legitimate. However, as only the formeroption offers completely diverse routes, this is therefore likely to bethe preferred binding.

The other case to consider is traffic between nodes on the same spanningtree or (often) pair of spanning trees. Consider the example of nodessubtended by core nodes 12 and 13 (<blue> and <red>). Traffic betweenthese nodes is constrained to run either on the paths defined by the<red> VID, or the <blue> VID; both are installed out from core nodes 12and 13 respectively. It can be seen that traffic within a specific STuses the same VID in both directions (i.e. same as bi-directional VID,as described earlier). In the example given, the availability of two STsand hence two VIDs offers alternate routes.

In each case the minimum number of VIDs required is two, e.g.assignments <purple>=10, <red>=10, <green>=12, <blue>=12 is one validassignment which obeys the rules given above; provided that the rulesare obeyed, core nodes may be assigned VIDs otherwise independently, tominimise their own VID usage under the particular routing constraints towhich they are subject.

The number of VIDs required increases whenever an alternative routeacross the core is required. Recall the discussion earlier which showedthat a VID defined the egress node from the core, and a single route tothat node from any other node, for example connection of <green> to<blue> (and vice-versa) defines a bi-directional route across the core.If it is desired to send some traffic from node 10 via link “x” toegress through node 12 its destination, an extra VID must be assigned toallow node 10 to steer the traffic between nodes 12 and 13. In thereverse direction an extra VID must also be assigned for a copy of the<green> ST rooted on 10, to allow node 12 to steer traffic between nodes11 and 10, because <purple> and <green> STs are not disjoint, and so theroute to be taken traffic for nodes on both STs cannot be disambiguatedby MAC address alone at core node 12.

The VID numerical values obey the same rules as earlier; two VIDs mustbe different unless the egress STs with which they are associated haveno nodes in common. Thus, the first extra VID to select the route from10 to 12 via “x” cannot be the same as that assigned either to <red> or<blue> because the extra VID duplicates the <red> one or egress fromnode 12. Similarly, the VID in the reverse direction on “x” cannot bethe same as that assigned either to <green> or <purple> because theextra VID duplicates the <green> one on egress from node 10. Statedanother way, if a single ST (in the ingress direction) is to be offeredD distinct core routes to a target “egress” ST, it must use D VIDs toselect the route, and these D VIDs must be distinct from the VID(s) usedby a other non-disjoint “ingress” ST to reach the same “target” egressST.

The process for allocating uni-directional VIDs can be summarised as:

-   -   allocate a VID to each of the C core nodes, which gives a        maximum of C (=4 in this example)    -   STs which are disjoint, or which touch only at core nodes, can        share the same VID because MAC addresses in each ST are then by        definition disjoint too (in this example giving a base        requirement for 2 VIDs only.)    -   allocate further VIDs to support route diversity D between ST        pairs; this requires D VIDs.

It can be seen that the upper bound of total VID use for this approachscales as C×D although the actual number of VIDs used can be less.

In summary, for the uni-directional case, with independently chosen VIDsin each direction, it is only necessary to ensure that the egress STsare not common. However, because a bi-directional VID is e2e (thedisjoint ST constraint applies simultaneously at both ends), it ispossible to reuse a VID only on two pairs of STs where no elements arecommon, i.e. where no node on either of the STs is common with any otherST.

The description above is of a network having a core network of nodes andsectors serving outlying nodes, with a pair of outlying nodes beingconnected by a concatenation of a segment between the first outlyingnode and a core node, a core network segment, and a segment between acore node and the second outlying node.

It is sometimes desirable to install ‘backdoors’ in a network to offer ashort-cut path between busy nodes without going through the networkcore. FIG. 9 shows an example backdoor path 70 between nodes 71, 72.This can be constructed by blocking core node. STs at the backdoor pathtransit node 73. This prevents unwanted extension of core node STs. Arestricted length ST is built, rooted on the backdoor path transit node73, using the same construction rules as core routes. This means thatthe ST stops short of any other core node. A bi-directional VID isallocated to that bypass ST.

The method described above is used to plan the provision of VIDs toroutes across the network with the goal of creating forwarding statewhich is both valid and minimised. Steps 61-63 of the method, i.e.selecting core nodes, and creating spanning trees, are tools which areused as part of the planning method. Once the planning method has beenperformed, routes are established within the network by instructing(step 64, FIG. 2) switches at the individual network nodes to storeforwarding state.

The above described method is used incrementally, multiple times. Onfirst deployment, one would expect to build the base set of routes tosatisfy resilience (diverse path) criteria, and load traffic onto thoseroutes. Then, if a new route through the existing network needed to becreated for traffic engineering reasons, one would create this acrossthe core by adding extra VIDs as appropriate, according to the samerules. Similarly, as new plant is added to the network (i.e. switches orphysical links), appropriate steps of the method are appliedincrementally as part of the process of bringing into service.

FIG. 10 schematically shows a number of switches 87 deployed in anetwork and a control plane of the network. A network planning system 80includes a processing system 81 which performs the method describedabove of analysing the network and allocating route identifiers (VIDs)in an optimum manner. Network topology data 82 is acquired either from alocal or remote store, or by interrogation of the network elementsthemselves, for use in analysing the network. Once an optimum allocationof identifiers has been determined, this data is forwarded to aconnection controller 85. The connection controller communicates witheach Ethernet switch 87 deployed in the network. Connection controller85 sends signalling information 86 to each switch 87 which instructs theswitch to store forwarding instructions to implement the allocationdetermined by processor 81. The instructions will take the form, forexample of: forward frames with a destination MAC_address1 and VID1arriving at port X of the switch to port Y of the switch, and forwardreturn-path frames with destination MAC_address2 and VID2 arriving atport Y to port X. The instructions are specific to each switch 87.Although a single connection controller (CC) is shown here, there can bea set of such controllers which are distributed throughout the network.

FIG. 11 shows an example of equipment at each managed Ethernet switch100. The switch 100 has a set of ports 110-117 which are interconnectedby a switch fabric 140. One such port 110 is shown in detail and eachport has the same structure. Ethernet ports are bi-directional andcomprise an ingress section 120 and an egress section 130 which share acommon MAC address. The ingress section 120 has a line interface 124which connects to a communication link 127. The line interface convertsthe incoming signal from a format used on the line 127 to a basebandelectrical digital signal which can be used by the following stages.Typically, this involves receiving, and demodulating, an optical signaland decoding the demodulated signal from a line code used fortransmission. For each frame, stage 125 determines a destination port ofthe switch that the frame should be sent to by examining fields withinthe frame and a priority for that frame. In the preferred scheme justdescribed, this requires stage 125 to inspect the destination addressfield and the VLAN tag field of the frame. A look-up table 123 is storedin a memory 122 local to the port. Stage 125 performs a look-upoperation using the destination address and VLAN tag to identify theport that the frame should be sent to. Stage 125 sends the frame to theappropriate buffer 126 where the frame is queued before the switchfabric transfers the frame to the required destination port. Typically,a switch will transfer fixed sized blocks of data across the switchfabric (e.g. cells 80 bytes in length) to maximise efficiency of theswitch and to minimise blocking between switch ports, but this is notimportant to the invention and the switch can use any technology for theswitch fabric. At the egress section of the destination port the blocksare reassembled in a buffer 131 before being sent to a line interface134. Typically, this will involve encoding the data frame with a linecode and modulating the electrical digital signal onto an opticalcarrier.

Each port is associated with a port controller 121 which is responsiblefor maintaining the forwarding table 123 at that port. Port controller121 communicates with a switch controller 102. Port controller 121 isessentially a ‘housekeeping’ processor which performs tasks locally atthe port, in response to instructions received from switch controller102. Switch controller 102 maintains a master forwarding table 104 instorage 103 local to the controller 102 and communicates via asignalling interface 105 to a connection controller. As connectionsacross network 5 are set-up, changed (e.g. due to traffic managementoperations) or torn down, switch controller 102 receives instructions toadd or remove entries in the forwarding table 104 from a networkconnection controller. The information received at switch controller 102from a Network Management System will typically refer to a physicaladdress and will set up a bi-directional path (i.e. forward and returnpaths) at the same time {i.e. forward frames with destinationMAC_address1 and VLAN1 arriving at port X to port Y; forward return-pathframes with destination MAC_address2 and VLAN2 arriving at port Y toport X}. Updated forwarding information is distributed by switchcontroller 102 to individual port controllers. In an alternative,simplified, switch the local forwarding table 123 at each port isomitted and only a single forwarding table 104 is used.

1. A method of planning routes and allocating route identifiers in amanaged frame-forwarding network, the network comprising a set of nodesinterconnected by links, with each node being arranged to forward dataframes according to a combination of a route identifier and a networkaddress carried by a received data frame and forwarding instructionsstored at the node, the method comprising: identifying a sub-set of thenodes which are core nodes of the network, the remaining nodes beingoutlying nodes; building a spanning tree off each of the identified corenodes, with the spanning tree stopping one link short of any other corenode, the spanning tree having a root and defining a loop-free pathbetween the core node at the root of the spanning tree and a set ofoutlying nodes; and, planning connections between roots of the spanningtrees and allocating a different route identifier to each plannedconnection between a pair of spanning trees.
 2. A method according toclaim 1 wherein planning the connections between roots of the spanningtrees plans a set of connections which minimizes the number of singlepoints of failure common to more than one different connection betweeneach pair of outlying nodes.
 3. A method according to claim 1 whereinbuilding the spanning tree off each of the identified core nodes isarranged such that, where an outlying node is served by spanning treesrooted on two different core nodes, the paths of the two spanning treesare differently directed to that outlying node.
 4. A method according toclaim 1 wherein identifying the core nodes in the network comprisesidentifying a core node as a node which is connected to at least N othernodes.
 5. A method according to claim 1 wherein the identifier is thesame for both directions of information transfer along a bi-directionalco-routed path between two nodes, and wherein planning the connectionsbetween the roots of pairs of the spanning trees connects each spanningtree with every other spanning tree and allocates a different routeidentifier to each planned connection.
 6. A method according to claim 5wherein allocating route identifiers must only allocate a differentroute identifier to a connection between a pair of spanning trees wherethe outlying nodes served by either of the spanning trees in the pairoverlap with any of the outlying nodes served by another spanning tree.7. A method according to claim 6 wherein the route identifier is anEthernet Virtual Local Area Network (VLAN) Identifier (ID) and thenetwork address is an Ethernet Destination Address.
 8. A method ofgenerating forwarding instructions for a node in a managedframe-forwarding network, the network comprising a set of nodesinterconnected by links, with each node being arranged to forward dataframes according to a combination of a route identifier and a networkaddress carried by a received data frame and forwarding instructionsstored at the node, the method comprising: planning routes andallocating identifiers in the managed frame-forwarding network by:identifying a sub-set of the nodes which are core nodes of the network,the remaining nodes being outlying nodes; building a spanning tree offeach of the identified core nodes, with the spanning tree stopping onelink short of any other core node, the spanning tree having a root anddefining a loop-free path between the core node at the root of thespanning tree and a set of outlying nodes; and, planning connectionsbetween roots of the spanning trees and allocating a different routeidentifier to each planned connection between a pair of spanning trees;and, generating forwarding instructions for a node based on the plannedroute and allocated identifier for that route.
 9. A method according toclaim 8 further comprising sending the generated forwarding instructionsto the node.
 10. A non-transitory machine-readable medium carryinginstructions for causing a processor to perform a method of planningroutes and allocating route identifiers in a managed frame-forwardingnetwork, the network comprising a plurality of nodes interconnected bylinks, with each node being arranged to forward data frames according toa combination of a route identifier and a network address carried by areceived data frame and forwarding instructions stored at the node, themethod comprising: identifying a sub-set of the nodes which are corenodes of the network, the remaining nodes being outlying nodes; buildinga spanning tree off each of the identified core nodes, with the spanningtree stopping one link short of any other core node, the spanning treehaving a root node and defining a loop-free path between a core node atthe root of the spanning tree and a set of outlying nodes; and, planningconnections between roots of the spanning trees and allocating adifferent route identifier to each planned connection between a pair ofspanning trees.
 11. A managed frame-forwarding network comprising: a setof nodes interconnected by links, with each node being arranged toforward data frames according to a combination of an identifier and anetwork address carried by a received data frame and forwardinginstructions stored at the node; a network planning entity for planningroutes and for allocating route identifiers to routes within thenetwork, the entity being arranged to: identify a sub-set of the nodeswhich are core nodes of the network, the remaining nodes being outlyingnodes; build a spanning tree off each of the identified core nodes, withthe spanning tree stopping one link short of any other core node, thespanning tree having a root node and defining a loop-free path betweenthe core node at the root of the spanning tree and a set of outlyingnodes; and, plan connections between roots of the spanning trees andallocate a different route identifier to each planned connection betweena pair of spanning trees; and a connection controller which is arrangedto generate forwarding instructions to cause the nodes to implement theroutes planned by the network planning entity.
 12. A method of planningroutes and allocating route identifiers in a managed frame-forwardingnetwork, the network comprising a set of nodes interconnected by links,with each node being arranged to forward data frames according to acombination of a route identifier and a network address carried by areceived data frame and forwarding instructions stored at the node, themethod comprising: identifying a sub-set of the nodes which are corenodes of the network, the remaining nodes being outlying nodes; buildinga plurality of spanning trees, wherein each spanning tree comprises oneof the core nodes at a root of the spanning tree, with the each spanningtree stopping one link short of any other of the core nodes, the eachspanning tree defining a loop-free path between the one of the corenodes at the root of the spanning tree and a set of outlying nodes,wherein at least some of the plurality of spanning trees are disjointfrom one another; and, planning connections between pairs of spanningtrees, and allocating a unidirectional ingress route identifier andunidirectional egress route identifier to each planned connection,wherein a same unidirectional egress route identifier is allocated to anegress route from a first spanning tree to multiple disjoint secondspanning trees.
 13. The method of claim 12, wherein the unidirectionalingress route identifiers and the unidirectional egress routeidentifiers comprise Virtual Local Area Network (VLAN) Identifiers (ID).